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SCHEDULING CONTROL FOR QUEUEING SYSTEMS WITH 
MANY SERVERS: ASYMPTOTIC OPTIMALITY 
IN HEAVY TRAFFIC 1 

By Rami Atar 
Technion-Israel Institute of Technology 

A multiclass queueing system is considered, with heterogeneous 
service stations, each consisting of many servers with identical capa- 
bilities. An optimal control problem is formulated, where the control 
corresponds to scheduling and routing, and the cost is a cumulative 
discounted functional of the system's state. We examine two versions 
of the problem: "nonpreemptive," where service is uninterruptible, 
and "preemptive," where service to a customer can be interrupted 
and then resumed, possibly at a different station. We study the prob- 
lem in the asymptotic heavy traffic regime proposed by Halfin and 
Whitt, in which the arrival rates and the number of servers at each 
station grow without bound. The two versions of the problem are not, 
in general, asymptotically equivalent in this regime, with the pre- 
emptive version showing an asymptotic behavior that is, in a sense, 
much simpler. Under appropriate assumptions on the structure of the 
system we show: (i) The value function for the preemptive problem 
converges to V, the value of a related diffusion control problem, (ii) 
The two versions of the problem are asymptotically equivalent, and 
in particular nonpreemptive policies can be constructed that asymp- 
totically achieve the value V. The construction of these policies is 
based on a Hamilton-Jacobi-Bellman equation associated with V. 

1. Introduction. We consider a queueing system with a fixed number of 
customer classes and a fixed number of service stations, each consisting of 
many servers with identical capabilities. Only some stations can offer service 
to each class, and the service rates depend on the class and on the station. 
Customers have a single service requirement and leave the system after it is 
met. They may also abandon while waiting to be served [see Figure 1(a)]. 
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Arrivals are modeled as renewal processes, while service and abandonments 
occur according to exponential clocks. Scheduling of jobs, which amounts 
to selecting what customer each server serves at each time, is considered as 
control. The goal is to minimize an expected cumulative discounted func- 
tional of general performance criteria, including queue lengths of different 
classes, number of customers in each of the stations and number of servers 
that are idle at each station. This paper analyzes the scheduling control (SC) 
problem in a central limit theorem (CLT) asymptotic regime. In particular, 
as proposed first by Halfin and Whitt [8], system parameters are rescaled in 
such a way that the arrival rates and the number of servers at each station 
are proportional to a parameter n, and the system is kept in heavy traffic 
as n becomes large. See [7] for motivation for this model and parametric 
regime, in particular in relation to call center models. 

One important feature of a CLT approach is that the non-Markovian 
structure of the SC problem becomes Markovian as weak limits are taken for- 
mally. In particular, a diffusion control (DC) problem arises in the limit, that 
is accessible by control-theoretic tools. In [2] we studied the dynamic pro- 
gramming partial differential equation (PDE) of Hamilton-Jacobi-Bellman 
(HJB) type for the DC problem and characterized the value function as its 
unique solution. It is natural to ask whether the value function for the SC 
problem converges to that of the DC problem. 

In fact, this question must be formulated more carefully. One could con- 
sider two versions of the problem (that formally yield two different DC 
problems): one in which only nonpreemptive scheduling is possible, and one 
where it is allowed to use preemptive policies, where service to a customer 
can be interrupted and then resumed, possibly at a different station (it is 
the DC problem associated with preemptive scheduling that was analyzed 
in [2]). Interestingly, these two versions show identical asymptotic behavior 
in some cases of the problem, and they show different behavior in other 
cases: In the single-station case, it was proved in [3] that, in both versions 
of the problem, the value function for the SC problem converges to that of 
the same DC problem. These results should be regarded as a simplification 
that shows in the limit, because the DC problem that is associated formally 
with nonpreemptive scheduling lies in higher dimension. On the other hand, 
we will present a heuristic argument suggesting that for a particular two- 
station system one does not expect the value function in both versions of 
the problem to have identical limits. Hence one would like to understand 
when the two versions of the problem behave the same asymptotically. 

To indicate what prevents the two-station example from undergoing the 
same simplification, we need the following terminology. A class-station pair 
is said to be an activity if servers of station j are capable of serving 
class-z customers [in Figure 1, (1,A) is an activity, and (2, A) is not]. A 
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distinction between two kinds of activities is made in terms of an under- 
lying static fluid model (introduced in Section 2): The fluid model has a 
positive fraction of the customer population in some activities and a zero 
fraction in others. Such activities are referred to as basic and, respectively, 
nonbasic activities (cf. [10]). Roughly, the population of customers in basic 
and nonbasic activities in the queueing model is typically 0(n) and, respec- 
tively, 0(n 1//2 ). Now, while under a heavy traffic assumption single-station 
systems have only basic activities; the two-station example alluded to above 
has a nonbasic activity. As explained in detail in Section 2.7, the simpli- 
fication will not occur in the presence of an activity having only 0(n 1 ^ 2 ) 
customers. Aiming at extending the results, that show simplification of the 
single-station nonpreemptive problem to the multistation setting, we are led 
to impose the assumption that, in fact, all activities are basic. In this paper 
we find that under this condition, a simplification indeed takes place. 

It follows from a result of Williams [14] that if all activities are basic, then 
the graph, having classes and stations as vertices and activities as edges, 
is necessarily a tree [as, e.g., in Figure 1(b)]. We refer to this structural 
condition by saying that the system is treelike. As in [2] , the results of this 
paper will therefore apply to treelike systems only. 

The main results are the following. Under appropriate assumptions, the 
preemptive SC problem's value converges to the value of the DC problem. 
Moreover, preemptive and nonpreemptive scheduling control policies (SCPs) 
are constructed, that asymptotically achieve the DC problem's value. These 
results establish the validity of the DC problem as the correct asymptotic 
description of the problem, as well as the asymptotic equivalence of the 
preemptive and nonpreemptive problems. 

In a preemptive SC problem, one can consider the vector representing the 
number of customers of each class present in the system as "state," and the 
vector representing the number of customers at each activity as "control" 
(at least in the Markovian case, i.e., under the assumption that arrivals are 
Poisson). In fact, "state" and "control" for the DC problem stand for the 
formal weak limits of precisely these quantities. The results of [2] regarding 
the DC problem assert that there exists an optimal Markov control pol- 
icy, sometimes referred to as an optimal feedback function. Namely, there 
is a function mapping state to control, that can be used to define the con- 
trol process as a feedback from the state process, in such a way that the 
DC problem's value is achieved. In this paper, the preemptive SC problem 
is solved, roughly, by dynamically setting the population in the activities 
equal to the image of the system's state under the feedback function for 
the DC problem. In the nonpreemptive version of the problem one does not 
have direct control over the population in the activities, and therefore a 
scheme as above cannot be applied. Instead one must affect the population 
in the activities indirectly by appropriate decisions of job allocations. To 
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this end, a kind of tracking mechanism is introduced, that roughly works 
as follows. One continuously computes the desired value of the population 
in all activities as the image of the state under the feedback function. One 
declares activities for which the actual population is greater than desired as 
"overpopulated." Overpopulated activities are then blocked in the sense that 
new jobs are only assigned through activities that are not overpopulated. 
As a result, the population decreases rapidly in overpopulated activities and 
increases rapidly in other activities. The main difficulty dealt with in this 
paper consists of estimating the difference between the desired and actual 
values so as to show that tracking is maintained. The proof requires two 
different types of arguments: one has to do with tightness of the processes 
involved, and the other regards estimates on their large time behavior. Al- 
though the methods of [3] are used in various places in the proof, the main 
difficulty alluded to above is far more complicated in the multistation case. 

We point out that in the more "standard" CLT regime, where the capac- 
ity of each server is scaled up rather than the number of servers, a diffusion 
model for a similar queueing system has been studied by Harrison and Lopez 
[10]. A reduction, quite different from the one alluded to above, takes place 
in this regime, that in fact makes the diffusion model one-dimensional. Re- 
cently, a number of works have developed policies that are asymptotically 
optimal in this regime, for models as in [10] as well as more general ones 
[1, 4, 13]. 

The organization of the paper is as follows. In Section 2 we introduce the 
probabilistic queueing model, the scaling and some assumptions regarding 
work conservation. We then describe the DC problem and the HJB equation, 
propose SCPs for the queueing model that are based on the HJB equation 
and state the main result regarding asymptotic optimality of these policies. 
Finally we discuss the roles of the main assumptions. Section 3 contains the 
proofs. 

Notation and terminology. Vectors in M. k are considered as column vec- 
tors. For x G K fc (resp., iel hi ) let ||x|| = Y^i\ x i\ (resp., J2ij\ x i,j\)- For 
two column vectors v, u, v ■ u denotes their scalar product. The symbols ej 
denote the unit coordinate vectors and e = (1, . . . , 1)'. The dimension of e 
may change from one expression to another, and for example, e ■ a + e ■ b = 
J2i a i + J2j bj even if a and b are of different dimension. Write N = {1, 2, . . .}, 
Z_|_ = {0, 1,2,.. .}, R + = [0, oo). The class of twice continuously differentiable 
functions on MJ is denoted by C 2 . The class of bounded functions in C 2 
is denoted by C 2 , and C po i denotes the class of functions / in C 2 sat- 
isfying a polynomial growth condition: there are constants c and r such 
that < c(l + ||:r|| r ), x € M 7 . For E a metric space, we denote by 

O(E) the space of all cadlag functions (i.e., right-continuous and having 
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left limits) from R + to E. We endow O(E) with the usual Skorohod topol- 
ogy (cf. [5]). If X n , n £ N, and X are processes with sample paths in 
O(E), we write X n =>• X to denote weak convergence of the measures in- 
duced by X n [on H)(E)] to the measure induced by X. For a collection 
A of random variables, denotes the sigma-field generated by this 

collection. If X is an R fc - or an R fcx '-valued process (or function on R + ) 
and < s < t < oo, ||-X"||* t = sup s<M<t ||A(u)||, and if X takes real values, 
\ x \t,t = sup s < u < t \X(u)\. Also, \\X]\l = \\X\\^ t and \X\$ = \X\^ t . The nota- 
tion X(t) and Xt are used interchangeably. For a locally integrable function 
/ : R + — ► R denote JP f = fQf(s)ds. In case that / is vector- or matrix- 
valued, JP f is understood elementwise. For y € R+ denote by [y] the element 
y' G Z k + having ^ = [yij for i = 1, . . . , fc- 1, and j/J. = Lz/fcJ + EiU(l/i ~ Lf*J)- 
Note that e • JyJ = e • y, and 

(i) l|y-Ij/]||<2fc. 

Finally, we need the following elementary graph-theoretic terms: A tree is 
an undirected, acyclic, connected graph; and the diameter of a graph is the 
distance between the two vertices that are farthest apart. 



2. Setting and results. 



2.1. Queueing model. The model under study has / customer classes and 
J service stations [see Figure 1(a)]. At each service station there are many 
independent servers of the same type. Each customer requires service only 
once and can be served indifferently by any server at the same station, but 
possibly at different rates at different stations. Only some stations can offer 
service to each class. When referring to the physical location of customers 
we say that they are in the buffer, or in the queue if they are not being 




Fig. 1. (a) A system with four classes and three stations, (b) The corresponding graph T. 
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served, and we say that they are in a certain station if they are served by 
a server of the corresponding type. There is one buffer per customer class 
and one station per server type. While a customer is in the queue it may 
abandon the system at rate depending on the class. 

A complete probability space (Q,F,P) is given, supporting all stochastic 
processes defined below. Expectation with respect to P is denoted by E. The 
processes will all be indexed by the parameter n € N, that will eventually be 
taken to infinity. The classes (buffers) are labelled as 1, . . . ,1 and the types 
(stations) as I + 1, ...,/ + J: 

T={1,. ..,/}, 

J = {I+1,...,I + J}. 

For j G J let iV" be the number of servers at station j. Let Xf(t) denote 
the total number of class- i customers in the system at time t. Let Y™(t) 
denote the number of class- i customers in the queue at time t. Let Zj l (t) 
denote the number of idle servers in station j at time t. And let V?£(£) denote 
the number of class-i customers in station j at time t. Let X n = (X")j S x, 
Y n = (17)i e x, Z n = (Z^) je j, = (*5)ieT,ie 1 7- Straightforward relations 
are expressed by the following equations, holding for all i & I, j £ J and 
t > 0: 



(2) TO + E *£■(*) 

(3) z;(t) + 5>™w = iV7, 

(4) Y?(t)>0, Z?(t)>0, 

(5) *?(*)> o, m(t) >o. 



To define arrival processes, let, for each iGl, {Ui(k),k G N} be a sequence 
of strictly positive i.i.d. random variables with EUi(l) = 1 and squared coef- 
ficient of variation (EUi{l))~ 2 Var(C/j(l)) = Cf Ji £ [0, oo). Assume also that 
the sequences are independent. Let 

(6) um=^u i (k), 

where \? > 0. With Ei = 0, define 

(7) ^(t) = sup|z>0:EC/f(fc)<t|, *>0. 

It is assumed that the number of arrivals up to time t is Af(t). Note that the 
first class-i customer arrives at J7"(l), and the time between the (k — l)st 
and kth arrival of class-i customers is U™{k). 
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To model service times as exponential independent random variables, let 
Sjj, i G J, be Poisson processes with rate /i^- G [0,oo) (where a zero- 

rate Poisson process is the zero process). These processes are assumed to be 
mutually independent, and independent of the arrival processes. Let T^(t) 
denote the time up to t devoted to a class-i customer by a server, summed 
over all type-j servers and note that 

Tp j (t) = jT*5-(s)d«, iei,jej,t>o. 

The number of service completions of class-z customers by all type-j servers 
by time t is S^{Tf-{t)). Note that this indeed models independent servers 
with exponential service times: Conditioned on = k, the time until the 
next service, when k servers are busy, is exponential with parameter propor- 
tional to k. Similarly, let Rf(t) be Poisson processes of rate 0" G [0, oo) and 

o 

let T2(t) denote the time up to t a class- i customer spends in the queue, 
summed over all customers. Then 

J o 

o 

and we assume that Rf(Tf (t)) class-i customers have abandoned until t. 
Throughout, the initial conditions, that we denote by X"> n :=Xf(0), are 
assumed to be deterministic. We have 

Xl\t) = X°* n + A^{t) - £ S% ^(s) da) - fl? (jT Y?(s) ds^j , 

(8) 

i G l,t > 0. 

The processes A n , S n and R n will collectively be referred to as the primitive 
processes. 

2.2. Scheduling. Scheduling decisions are made by continuously select- 
ing subject to appropriate constraints. Scheduling is regarded as pre- 
emptive if service to a customer can be stopped and resumed at a later time, 
possibly in a different station. Formally this is expressed by stating that the 
process \I/ may be selected subject only to (2)-(12) holding. Note that ac- 
cording to this definition, customers can be moved instantaneously not only 
between a service station and the buffer, but also between different service 
stations that offer service to the corresponding class. Scheduling is regarded 
as nonpreemptive if every customer completes service with the server it is 
first assigned. More precisely, consider the processes B^(t), i 6 J, 

where B™A0) = 0, and Bf, increases by k each time k class-i jobs are moved 
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to station j from the buffer or from another station (to start or resume ser- 
vice), and decreases by k each time k such jobs are moved from station j 
back to the buffer or to another station. Then can be expressed as 



To define nonpreemptive scheduling in terms of the model equations (2)- 
(12), we will require that \& is selected subject to (2)-(12) and such that Bf^ 
are nondecreasing processes. This is summarized in the following. 

Definition 1. Let initial data X°' n and primitive processes A n , S n and 
R n be given. 

(i) We say that ^f n is a preemptive resume scheduling control policy 
(P-SCP) if Vl/jj have cadlag paths (i G X, j £ J\ and there exist processes 
X n ,Y n and Z n such that (2)-(12) are met. X n is said to be the controlled 
process associated with initial data X 0,n and P-SCP ty n . 

(ii) We say that *$> n is a nonpreemptive scheduling control policy (N-SCP) 
if it is a P-SCP and Bfj of (9) have nondecreasing paths. 

We collectively refer to P-SCPs and N-SCPs as scheduling control policies 



A word on terminology: The reader may notice that these definitions 
make nonpreemptive policies a special case of preemptive ones, unlike the 
usual use of these terms in the literature, where they are mutually exclusive. 
Indeed, the term "preemptive" is usually used to indicate that service to a 
customer is necessarily interrupted if a customer of higher priority arrives, 
whereas "nonpreemptive" is used when interruptions are not allowed. In the 
current context, however, the term "preemptive" indicates only that it is 
possible to interrupt service to a customer, and so it is natural that a policy 
under which no interruption occurs is regarded as a special case. 

Let C = (X°' n ;n£ N) andp=(f ! ,n6 N) denote a sequence of initial 
conditions, and respectively, SCPs. We denote by Pj? the measure under 
which, for each n, X n is the controlled process associated with X°' n and 
\P n . Expectation under P^ is denoted by E^. 

We need a notion of SCPs that do not anticipate the future. Unlike in a 
Markovian setting, in the presence of renewal processes such a notion has 
to take into account that the time of the next arrival is correlated with 
information from the past, and hence should not be regarded as innovative 
information (see also [3]). Denote 



(9) 




(SCPs). 



7f (t) = inf{« > t : A?(u) - A?(u-) > 0} 
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= a{A2(s),Sr j (Tr j (s))^(TUs)),n(s), 

(10) 

X?{s), Yi(s), Zj(s) :iel,jej,s<t} 

and 

G? = o{A?{T?{t) + u)-A?{T?(t)l 

(11) ssW) + «)-s5W)), 

fl?(T?(t) + u) - fl?(T?(t)) : i G X, J G J, « > 0}. 

Definition 2. We say that a scheduling control policy is admissible if: 

(i) for each t, is independent of Qf; 

(ii) for each i, j and t, the process Sg-(I$(t) + ■) - S^(T^(t)) [resp., 
R?(T?(t) + ■)- R?(T?(t))} is equal in law to Sg.(-) [Ef (■)]. 

2.3. Fluid and diffusion scaling. We now introduce some assumptions 
regarding the asymptotic behavior of the parameters as n — > oo. To this 
end, consider the graph T having vertex set IU J with a vertex for each 
class and a vertex for each type, and an edge set 8, with an edge joining a 
class and a type if the corresponding service rate is nonzero: 

S = {(i,j)elxj:^.>0}. 

It is assumed that the graph does not depend on the parameter n. We denote 
i ~ j and j ~ i if (i, j) S £. By assumption we have 

(12) ¥& = 0, i^j. 

A buffer-station pair (z, j) is said to be an activity if (i,j) G £. 

Scaling of parameters. There are constants Aj > 0, 9i > 0, i G X, and 
Mij > 0, £lx J where /iy > whenever i ~ j, such that 

(13) n" 1 *?-^, A$-A*ii, — 0i 
and 

Note that /iy = for i 96 j, since we had = for i 7^ j. Setting 

l^ij — Vj fiy , i £E X, j ' € JT" , 

a central assumption on the limit parameters, indicating that the sequence 
of systems is asymptotically critically loaded, is stated below (cf. [9, 10]). 
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(15) 



(14) 




J€ J 



iei. 



(16) 



iij > 0, i 



iel,jej. 



Heavy traffic condition. There exists a unique optimal solution (£*,/£>*) to 
the linear program. Moreover, ^iex^w = 1 f° r au 3^3 (and, consequently, 



In the rest of this paper, denotes the quantities from the above condi- 
tion, and x* = (x*), tp* = (tp*j), where 



We refer to the quantities £*j , x* and as the static fluid model. 

In a fluid model where class-i customers are served at station j at rate jlij , 
the quantities J2j QjP'ij represent the actual service rate of class-i customers 
when each station j allocates a fraction of its servers to class-i jobs. The 
quantities x* represent the total mass of class-i customers in the system. The 
heavy traffic condition indeed asserts that the system is critically loaded in 
the following sense. On one hand, (14) states that customers arriving at rates 
Aj can be processed by the system. On the other hand, under this condition 
it is impossible to have a £ satisfying (15) (with p < 1) and (16), and a 
A 7^ A, Xi > Xi, i £ I, such that (14) holds for p,, X and £ (since this would 
violate the uniqueness of £*). Thus, the system cannot process arrivals at 
rates greater than Aj. 

Following terminology from [10], an activity (i, j) £ £ is said to be basic 
if & > 0. This indicates that under the allocation matrix £*, class-i arrivals 
are actually processed by station j in the fluid model. In this paper we 
assume that, in fact, all activities are basic. More precisely, we will assume: 

Assumption 1. The heavy traffic condition holds, the graph T is con- 
nected and all activities are basic [i.e., > for all (i, j) G £]. 

By a result of Williams ([14], Theorem 5.3 and Corollary 5.4), Assump- 
tion 1 implies that 

(18) the graph T is a tree. 

As a result, Assumption 1 can equivalently be stated as: The heavy traffic 
condition holds, the graph T is a tree and all activities are basic. The as- 
sumption that T is a tree was exploited in the PDE analysis in [2] . We have 
more to say about Assumption 1 in Section 2.7. 
Second-order scaling assumptions follows. 
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Scaling of parameters, continued. There are constants Aj, fa € R, i £ X, 
j £ J , such that 

(19) Ar-nV^n^Af-A^^Ai, Ay : = ~ /%) - /%> 

(20) n 1 / 2 ^- 1 ^ -!/_,-) -^0. 

Scaling of initial conditions. There are constants Xj, y,, 2,-, satisfying 
Ui + J2j i>ij = Xi, Zj + J2i 4>ij = 0, yi> 0, and Zj > 0, i£l, j £ J, such that 
the deterministic initial conditions satisfy 

(21) 





:= n 


-l/2 (x 0,n 


- nxi ) - 




y0,n 

i 


:= ra~ 


■l/2y0,n _ 

t 






iy0,n 

Z j 


:= ra~ 


-1/2 ^0,n _ 


-> Zj, 




»J 


:= ra~ 


1/2^0," 


- - 





(22) 

The processes rescaled at the fluid level are defined as 
X?(t) = n^X^t), Y t n {t) = n-X n (t)i 
Z™{t)=n- l Z"{t), mt) = n- 1 *m 

The primitive processes are centered about their means and rescaled at the 
diffusion level as 





AUt)=n- 1 / 2 (AUt)-\?t), 


(23) 


S%(t) = n- l / 2 (S? j (nt)-ntf j t), 
R^(t) = n~ l / 2 {R^{nt) - n9ft). 


Similarly, the state processes are centered about the static fluid model and 
rescaled: 


(24) 


X?{t) = n- 1 ' 2 (X?(t)-nx* i ), 


(25) 


Y l n it)=n- 1 ' 2 Y l n (t), 
Z™(t)=n- 1 / 2 Z™{t), 


(26) 


^•(t) = n- 1/2 TO*)-W- 


The relations (2), 


(3) and (4) take the new form 


(27) 


3 


(28) 


i 


(29) 
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Using the definitions above one finds that (8) can be written as follows (for 
the reader's convenience this development is provided in the Appendix): 

(30) x?(t) = if n + nw?(t) + ^-E tfj jl ds ~ e ? [ ^ n ( s ) ds > 



where 



(31) 



With (19) we have 

(32) lim£2=£i:=k-J2^ij- 

j 

One is free to choose the values of r^, and it will be convenient to choose 
them so that, with the formal substitution = ^tji = 0, one has 

(33) limE[{W?(l)) 2 ] = l. 

Namely, n = (AjC^ + Xi) 1/2 . Denote also £ = (£ u . . .,£i)', r = diag(r,). 

The results of this paper are concerned with constructing sequences of 
SCPs that, in an appropriate sense, minimize the limit as n — > oo of cost of 
the following form: 

roc 

(34) E / e-^L(X^,^)dt. 

Jo 

2.4. Joint work conservation. A policy is said to be work conserving if 
it does not allow for a server to idle while a customer that it can serve is in 
the queue. We can express this condition as 

Y™(t) A Z'jit) = Vi~j,t>0. 

In the current context one can consider a stronger condition for preemptive 
policies. Recall that if preemption is allowed, customers of each class can be 
moved between the queue and the various stations that offer service to them. 
In a preemptive policy, controls ^f n (t) that can be applied at time t corre- 
spond to different rearrangements of the customers X n {t) in the stations 
and buffers. Let X n denote the set of all possible values of X n {t) for which 
there is a rearrangement of customers with the property: either there are no 
customers in the queue, or no server in the system is idle. This property is 
expressed as 

(35) e-Y n (t)Ae-Z n (t) = 0. 
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We shall say that a preemptive policy is jointly work conserving if it is work 
conserving and, in addition, for every t, if X n (t) £ X n , then customers are 
arranged according to (35). Some motivation is provided in Section 2.7. 
Let ao > denote the constant 

(36) a = (4C G )- 1 min if,* 

1,3 : t~j 

It will be shown (see Lemma 3) that 

(37) || X n (t) - Tix* || < a n implies X n (t ) £ X n 

(note that the value of t is irrelevant in the discussion). This fact is sig- 
nificant for the following reason. Under appropriate conditions, it will be 
established that the processes X n are tight. As a result, the condition on 
the left-hand side of (37) holds for every t £ [0,T] (for arbitrary T), with 
probability approaching 1 as n — > oo. Thus it is nearly always the case that 
a rearrangement of customers according to (35) is possible. These details 
will be justified in the process of proving our result. 



2.5. The diffusion control problem. We take limits as n — > oo in (27), 
(28), (30) and (35). The process W n converges to a standard Brownian mo- 
tion, and denoting the weak limit of (X n ,Y n , Z n , W n , if n ) by (X, Y, Z, W, *), 
we obtain the equation below (at this point this is meant as a formal step 
only; however, see Proposition 3): 

(38) Xi(t) =Xi + Wiit) -2llHj / ds-Oi Yi{s) ds, iel, 

j Jo Jo 

where Wi{t) =riWi(t) +£it, W is a standard Brownian motion and 

(39) ^2* ij =X i -Y i , iel, 

3 

(40) ]T*y = jej, 

i 

(41) e-YAe-Z = 0. 

Relations (38)-(41) above can be written in the convenient form 

dX = b(X, U)dt + rdW. 

To this end, note that by (39) and (40), e ■ X = e ■ Y — e ■ Z, and thus by 
(41) 



e-Y = (e-X) + , e-Z = (e-X)~ 
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Hence Y and Z can be represented in terms of the process e • X and an 
additional process U as 

(42) Y l (t) = (e-X(t)) + u i (t), Z j (t) = (e-X(t))- Vj (t), ieljej, 
where U(t) = (u(t),v(t)) takes values in 

U :={(«,«) GK J+J :iti,^ >0,iGX,j £j, e-u = e-v = l}. 
The following is shown in Proposition 7 of [2]. 

Lemma 1. Let Assumption 1 /ioW. Then given cti,/3j 6R, i GX, j G iX 
satisfying e ■ a = e ■ j3, there exists a unique solution tp = (ipij,i G X, j G JT) G 
M 7,7 to toe set o/ equations 

j » 

(43) 

ipij =0 /or i rjC j. 

As a result there is a map, denoted throughout by G:Dq — > 
D G :={(a,/3) GM /+J :e-a = e-/3}, 
such that ip = G(a,P). Let also 

(44) C G = sup(max|Gy(a,/?)| : (a,/?) G £> G , ||a|| V \\/3\\ < 1 

Clearly the map is linear on Dq- Applying Lemma 1 to (39), (40), and 
by (42) it follows that 

(45) * = G(X - Y, -Z) = G{X - (e • X)+u, -(e • X)~v) =: G(X, U). 
Letting 

(46) bi(X, U) = -J2 IHjGijiX, U) - 0i(e ■ X)+ Ui + £ h 
we can now write (38) as 

(47) X{t) = x + rW{t) + b(X(s),U(s))ds, 0<t<oo. 

Jo 

We summarize the equivalence between the two representations in what 
follows. 



Lemma 2. // (38)-(41) hold, then (47) holds with some U -valued U. 
Conversely, if (47) holds (with U taking values in V), then one can find 
V,Y,Z such that (38)-(41) hold. In both cases, W(t) =rW(t)+£t. 
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Proof. We have already proved the first statement of the result. To 
see the converse, write U = (u,v), let Y = (e • X) + u, Z = (e • X)~v, and 
^ = G(X — Y, —Z). Equations (39), (40) automatically hold, and (45) implies 
(38). □ 

Definition 3. We call tt = (O, F, (F t ),P, U, W) an admissible system 
if (fl, F, (F t ), P) is a complete filtered probability space, U is a U-valued, 
(-Ft)-progressively measurable process, and W is a standard /-dimensional 
(-Ff)-Brownian motion. The process U is said to be a control associated with 
tt. X is said to be a controlled process associated with initial data x £ M. 
and an admissible system it, if it is a continuous sample paths, (i^)-adapted 
process such that J \b(X(s), U(s))\ ds < oo, t > 0, P-a.s., and (47) holds 
P-a.s. 

For any and any admissible system tt there exists a controlled 

process X, unique in the strong sense (cf. [2]). With an abuse of notation 
we sometimes denote the dependence on x and tt by writing P£ in place of 
P and E^ in place of E. We denote by II the class of all admissible systems. 

Given a constant 7 > and a function L, the cost of interest for the 
queueing system is given by (34). It is convenient to perform change of 
variables from (X, to (X, U). To this end, define L as 



Our conditions on L (given mostly via conditions on L) are stated below. 
They were required to carry out the PDE analysis in [2]. 

Assumption 2. (i) L(x, U) > 0, (x, U) e R 1 x U. 

(ii) The mapping (x,ip) 1— ► L(x,ip) is continuous. In particular, the map- 
ping (x, U) 1 ^ L(x, U) is continuous. 

(hi) There is g £ (0,1) such that for any compact icl^, 



holds for U € U and x,y £ A, where c depends only on A. 

(iv) There are constants c > and mj, > 1 such that L(x,U) < c(l + 
\\x\\ mL ), {JeU, xgR 1 . 

Consider the cost 



L(X, U) = L(X, G(X - (e ■ X) + u,-(e • X)~v)) 



(48) 



X£R I ,U = (u,v)eU. 



L(x,U)-L(y,U)\<c\\x-y 




x £ R 1 , 7T 6 II. 
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Define the value function as 

V{x)= inf C(x,tt). 

7rGlI 

The HJB equation for the problem is 

(49) Cf + H(x,Df)-jf = 0, 

where D denotes the gradient, C = (1/2) J2i r i d 2 /dxf, and 

(50) H(x,p)= mf [6(x, U) ■ p + L(x, U)] . 

The equation is considered on R with the growth condition 

(51) 3C,m \f(x)\ <C(1 + ||x|| m ), xeR 1 . 

Definition 4. Let x G 1^ be given. We say that a measurable function 
h : M. 1 — > U is a Markov control policy if there is an admissible system tt and a 
controlled process X corresponding to x and n, such that U s = h(X s ), s > 0, 
P-a.s. We say that an admissible system ir is optimal for x, if V(x) = C(x, tt). 
We say that a Markov control policy is optimal for x if at least one of the 
admissible systems corresponding to it is optimal. 

We note in passing that the last part of the above definition is equiv- 
alent to optimality of all corresponding admissible systems, due to weak 
uniqueness of solutions to X(t) = x + rW(t) + J ' b(X(s), h(X(s))) ds (cf. 
Proposition 5.3.10 of [11]). 

The assumption below is needed for large time estimates on the state 
processes. Unless L is bounded [as in part (hi)], such estimates are required 
for both the DC problem and the SC problem, and they are only known 
to hold under parts (i) and (ii) below. This is explained in the remark in 
Section 3.3. 

Assumption 3. Either (i), (ii) or (hi) below holds. 

(i) For (i,j) £ £, fJ-ij depends only on i; or for (i,j) G £, ^ij depends 
only on j. In addition, 0j = 0, i El. 

(ii) The tree T is of diameter 3 at most. In addition, for all G £, 
®i < IHj ■ 

(hi) The function L is bounded. 
The following is proved in [2]. 

Theorem 1. Let Assumptions 1, 2 and 3 hold. Then the value V solves 
(49), (51), and there exists a Markov control policy h* '.R 1 — > U that is 
optimal for all x G W 1 . In cases (i) and (ii) [resp., (hi)] of Assumption 3, 
uniqueness holds in C^ ol [resp., C%\. 
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2.6. Asymptotically optimal SCPs. To state our main result we need to 
introduce SCPs defined via the function h* of Theorem 1. Write h* = {h\,K£) 
where (u,v) = h*(x) 44> u = h\{x),v = h^ix). Theorem 1 states that, for the 
DC problem, there is an optimal pair (U,X) for which U(t) = h*(X(t)), or 
equivalents u(t) = h\(X(t)), v(t) = h* 2 {X{t)). In view of (42) and (45), this 
can be written as 

Y(t) = (e ■ X(t)) + ht(X(t)), Z(t) = (e ■ X (*))" h*(X (s)) , 

m = G(X(t)-Y(t)-Z(t)). 

Let us define 

(52) Y n (t) = (e ■ X n {t)) + h\(X n {t)), Z n (t) = (e • X n (t))~ h* 2 (X n (t)) , 

(53) 4> n (t) = G{X n {t) - Y n (t), -Z n {t)). 

In analogy with the DC problem, one would like to define a P-SCP simply 
by setting 

(54) ^ n (t) = 4> n (t). 

However, this is impossible, because the components of the corresponding 
process ^ n (t) = ij)*n + n 1//2 ^ n (t) [cf. (26)] represent number of customers, 
and must be Z+-valued. Instead, a P-SCP will be defined in a way that the 
equality (54) will hold "roughly" To this end, consider two cases. 

Case 1. If \\X n (t) — nx*\\ <aon, set 
Y n {t) = {n 1 ' 2 Y n (t)l 

(55) Z n {t) = {n 1 ' 2 Z n {t)l 

^ n {t) = G(X n {t) - Y n (t),N n - Z n (t)). 

Case 2. If ||X n (t) - nx*\\ > a n, set ^> n = F n (X n (t)) where, for each n, 
F n is a fixed function chosen in such a way that the resulting process ^ n 
is jointly work conserving (as defined in Section 2.4). Other than that the 
choice of F n is immaterial. 

The choice of ^ n {t) in Case 1 is in accordance with joint work conserva- 
tion. Namely, (35) always holds on {||X n (£) — nx*\\ < aon}. To see this note 
that by (52), e • Y n (t) A e • Z n (t) = 0, hence by (55), e • Y n (t) A e • Z n (t) = 0. 
Also, since by definition X n ,Y n and Z n are integer valued, so is *$> n as fol- 
lows from the proof of Proposition 1 of [2] . It remains to show that setting 
& n (t) as in (55) meets relation (5) of the model, namely that ^fj(t) > for 
all i,j. The proof of this fact is deferred to Section 3.1 (cf. Lemma 3 and 
the remark that follows). Denote the resulting sequence of P-SCPs by p*. 
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The proposed N-SCP is described next. The idea is still to keep close 
to \fr n , but it is no longer possible to assign the values of ^ n in a direct way: 
*f> n is indirectly affected by the job assignments. Heuristically, the policy we 
propose is a kind of tracking mechanism, in which ty n tracks \& n , as follows: 

(a) Compute tf n (t) via (52), (53). 

(b) Declare activities £ £ with $fj(t) > ^fj(t) as "overpopulated." 

(c) Block overpopulated activities. That is to say that no new job as- 
signments are allowed on an activity as long as it is overpopulated. As a 
result, it is anticipated that the population in these activities drops rapidly, 
and the population in the other activities increases rapidly. 

(d) Obtain ^ n (t)~^ n (t). 

We turn to the precise definition of the N-SCP. For a technical reason, we 
must change the definition (52) of Y n (t) and Z n (t) so that the function h* is 
replaced by another function for large values of t. Namely, let h = (h®, h®) 
where hi(x) = e± and /i[>(x) = e/ + i for all x. Instead of (52), we let Y n (t) 
and Z n {t) be defined as 

Y n (t) = (e-X n (t)) + h 1 (t,X n (t)), Z n (t) = (e-X n (t))-h 2 (t,X n (t)), 

where 



(56) h(t,x) = (h 1 (t,x),h 2 (t,x)) 



n - 



(h*(x), t<e 
\h°(x), t>e n 



and n is a random time to be defined below. The definition of ^f n is still 
via (53). 

Next, we would like the following to hold. Given an activity and a 
time interval [s,t), if > holds on [s,t), then no routings take place 
on the activity throughout this interval: 

(57) i/fj > on [s,t) implies is constant on [s,t). 

On the other hand, when there is a class-i customer in the queue and there 
are stations j ~i with idle servers and such that is not blocked, the 
customer is instantaneously routed to one of these stations. If there are no 
such stations, the customer stays in the queue. It is not hard to see that this 
property can be expressed as follows. For every activity 

(58) mt)<%(t) implies Y™(t) A Z"'(t) = 0. 

The selection of an activity among the nonblocked activities through which 
to route a customer does not turn out to be important. However, care must 
be taken when two (or more) customers are routed instantaneously. There- 
fore it remains to show that one can always perform instantaneous routings 
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meeting (58). This is deferred to Section 3.1 (cf. Lemma 4). Finally, we 
define n as 

(59) 9 n = inf St : max[^(t_) - i^(l n (i_))] > 6 }, 
where 

F*(x) = G(x - (e • x) + h\{x), -(e • x)~h* 2 (x)), 
and bo denotes the deterministic, finite constant 

(60) b = 2 + maxsup(*£(0) - <&™(0)) + . 

i~j n 

A sequence of N-SCPs satisfying the above properties will be denoted by 
p' . We will also consider a sequence of N-SCPs defined exactly as above, 
except that for each n, the function h* will be replaced by a function h n , 
depending on n. Given a sequence {h n } of functions h n : M. 1 — > U, n £ N, we 
denote the corresponding sequence of N-SCPs as p'({h n }). 

The estimates used to establish asymptotic optimality will require some 
assumptions on the inter arrival times. The two parts of the assumption 
below correspond to different parts of the result. 

Assumption 4. There is a constant m\ such that E(Ui(l)) mA < oo, 
i £ X, satisfying either of the following: 

(i) rriA > 2m,£ (where m,L is as in Assumption 2); 

(ii) mA{mA — 2)(5m J 4 — 2)" 1 > m^. 

For a sequence £ of initial conditions and a sequence p of SCPs, denote 

POO 

Z(C,p) = liminf^ / e-^L(X?,^)dt, 
F(C,p) = limsup£? / e-^L(X?,4>?)dt. 

n— >oo JO 

Our main result is the following. 

Theorem 2. Lei Assumptions 1, 2 and 3 hold. Let be a sequence of 
initial conditions (X°' n ;n £ N) snc/i i/iaf A 0>n = ra _1 / 2 (A 0,n — rax*) — > x £ 
R . T/ien items (i) and (ii) below hold under Assumption 4(i), and items 
(iii) and (iv) ZioW under Assumption 4(ii). 

(i) For any sequence p of jointly work-conserving admissible P-SCPs, 
V(C,p)>V(x). 

(ii) Tae sequence p* of jointly work-conserving admissible P-SCPs sat- 
isfies V((,p*)<V(x). 
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(iii) Provided that h* ( of Theorem 1) is locally Holder on {£ £ M 1 : e ■ £ 7^ 
0}, the sequence p' of admissible N-SCPs satisfies V((,p') < V(x). 

(iv) Provided that the mapping U 1— ► [/) is convex on U /or every 
£ £ R 7 , i/iere exists a sequence {h n } of functions mapping M. 1 to U suc/i t/icrf 
£/ie corresponding sequence p" := j/({/i n }) 0/ admissible N-SCPs satisfies 
V((,p")<V(x). 

The combination of items (i) and (ii) establishes the asymptotic optimality 
of the sequence p* , as well as the validity of the DC problem as the correct 
asymptotic description of the problem under preemption. Items (i), (iii) and 
(iv) establish similar consequences for the nonpreemptive problem. 

To demonstrate the horizon of the different parts of the result, we men- 
tion some possible cost functions associated with queue length, and note in 
passing that one can also consider, for example, costs associated with the 
number of idle servers, and, via a transformation developed in [3], abandon- 
ment count. 

Recall that Y™ denote normalized queue lengths, and that Y™ = (e • 
X n ) + u™. For simplicity write these quantities as Yi, Xi and U{. Then a 
cost of the form J2i c «^7\ where q, a > 0, can be written as 

L(X,U) = ((e-X) + ) a J2ciU?. 

i 

We see that Assumption 2 is met and therefore items (i) and (ii) of Theo- 
rem 2 are in force. For item (iii), it follows from Proposition 3 of [3] that the 
Holder hypothesis on h* is met for the above cost with a > 2. For item (iv), 
clearly a > 1 suffices. Next, our treatment of the case where L is bounded 
[cf. Assumption 3 (iii)] allows to relax much of what is assumed otherwise 
[i.e., Assumption 3(i) or (ii)]. A truncated version of the above cost function, 
that is, 



L(X,U) 



((e-*) + r£ 



CM] 



AM, 



is an example in which items (i) and (ii) apply under Assumption 3(iii). 
As far as item (iv) is concerned, the convexity condition imposes the choice 
a = 1 and c = e, which amounts to the truncated sum of the queue lengths 
(e • Y) A M, expressed as 

L(X, U) = (e- X) + A M. 

Clearly, any other positive, bounded Holder function of X meets the as- 
sumptions of item (iv). 

As a final comment we mention that the PDE (49), on which the above 
proposed SCPs depend, can be solved numerically. The complexity of numer- 
ical schemes depend mainly on the dimension of the underlying Euclidean 
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space. The PDE (49) is of dimension I, and to realize the dimensionality 
reduction obtained by our results as far as nonpreemptive SC problems are 
concerned, note that the state process for such a problem in the Markovian 
case should consist of (Y n ,\E' n ). Since the number of activities under the 
treelike assumption is equal to I + J — 1, the reduction is from dimension 
21 + J - 1 to /. 

2.7. Discussion. We provide some motivation on two central assump- 
tions that we have made. First, we present an example that demonstrates 
heuristically why, in presence of nonbasic activities, the limit behavior is 
expected to be different for preemptive and nonpreemptive scheduling. The 
example consists of the simplest system having a nonbasic activity. The 
observation and example are due to Mandelbaum and Reiman [12]. 

Example 1. Consider a system with two customer classes and two 
server types as depicted in Figure 2. Servers of type A can serve both 
classes and servers of type B can only serve class-2 customers (in the ex- 
amples we refer to stations as A,B, ... rather than as numbers in J). Con- 
sider first a case where all activities are basic. For example, let u\ = V2 = l, 
fJ>lA = f^2A = ^2B = 1) and Ai = 1/2, A2 = 3/2. Then there is a unique op- 
timal solution to the linear program, namely: (S,ia^2A^2b) = (1/2)1/2,1). 
In a fluid model this means that half of the servers at station A serve class 1 
and all other servers serve class 2. In the queueing model with large n, under 
any policy that keeps the deviations from the fluid model at the desired level 
0(y/n), the number of class-1 and the number of class-2 customers in sta- 
tion A are ^1^, ^2 A = n /2 + 0(y/n), while the number of class-2 customers 
in station B is ^2B = n + 0{yfn). In a system with preemption the policy 
dynamically selects the values of these processes \& n . The control ^ that 
appears in the diffusion model shows up as a weak limit of the appropri- 
ately normalized, centered versions of the ^f n process. It therefore reflects 

I I 



1 I I 2 




I I 

Fig. 2. An example with three activities. 
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the 0(y/n) deviations referred to above. Now, in order for a system with- 
out preemption to be governed by the same diffusion model, it should be 
possible for ^ n to track closely the diffusion control process if . In partic- 
ular, it should be possible for the population in any activity to change by 
0(y/n) in o(l) units of time (this is the heuristic behind the proofs carried 
out in Section 3). Recall that the population in this activity is about n/2. 
If routing to this activity is stopped, its population will decrease due to 
service completions by 0(y/n) in o(l) units of time. On the other hand, if 
routing to this activity is maintained and routing to the other activities is 
stopped, the population at activity (2, A) will build up by 0(y/n) in o(l) 
units of time. This demonstrates how to rapidly decrease or increase the 
population in this activity, so as to allow *$> n to track a desired control pro- 
cess. Consider now a case where [i and v are as before, but Ai = A2 = 1. 
The unique solution to the linear program is now (^ia^2A^2b) = (IjOj 1)> 
and therefore activity (2, A) is nonbasic. The queueing model will have only 
0(y/n) customers in this activity. As a result, it is impossible to reduce the 
population in this activity in o(l) units of time so as to maintain a "good" 
tracking mechanism. This stands in contrast with the preemptive problem, 
where customers can be moved instantaneously between the buffer and the 
stations that offer them service. 

We next argue that joint work conservation is in many cases a desired 
property for problems with preemption (for problems without preemption, 
work conservation is typically not optimal, cf. [3], but under appropriate 
conditions the distinction between the two disappears in the limit, as seen 
in our main result). We do not attempt a rigorous treatment, and instead 
describe an argument towards optimality in a particular case. We comment 
that even in the context of one station, proving that work conservation is 
optimal is not trivial (see [3]). However, it is believed that the argument 
below can be greatly generalized. 

Example 2. The example consists of a Markovian system with two 
customer classes, two server types and no abandonment (see Figure 3). In 
(a) there is a customer of class 1 in the queue and a free server of type 
B. Although the free server cannot serve this customer, a rearrangement is 
possible so as to allow all customers to be served (b). We shall argue that 
"good" preemptive policies prefer option (b) over (a). Clearly this question 
must be coupled with the cost criteria. For concreteness, consider a cost 
of the form J2 c i^i P er urn t time (weighted sum of queue lengths). Use the 
fact that optimality is obtained by feedback policies, that observe only the 
state: number of customers at each class present in the system. Given a 
feedback policy tt that leaves a customer in queue 1 when there is a free 
server in station B, we show there exists a policy n that pays a smaller 
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Fig. 3. (a) A class-1 customer waiting in the queue, no free servers in station A and 
one free server in station B. (b) A class-2 customer moved from station A to station B 
and the class-1 customer moved from the queue to station A. 



cost on average. The argument is valid if \x^A < max(/ii J 4, ^2b)- Arguing by 
coupling and assuming the system starts at the described state as time zero, 
let tt move a class-2 customer, C2, from station A to station B and let it 
move the class-1 customer waiting in the queue, C±, into station A, until, 
at time r, there is an arrival or a service in one of the systems (i.e., the 
system under tt and that under it). If the first to occur is an arrival or a 
service to a customer other than C\ or C2, vr switches back to act like tt for 
all times. One can perform the coupling in such a way that service to C\ 
under tt always occurs later than the first between service to C\ and service 
to C2 under ir. If either service to C\ or to C2 under it is the first to occur, 
it then mimics tt except for the single customer that under it is not present 
and may still be present for a while under tt. In all cases the cost paid by it 
is not greater than that paid by tt. 

3. Proofs. 

3.1. Preliminary results. In Section 2 we obtained the convenient repre- 
sentation (47) of the controlled process from (38)-(41). We begin by showing 
how an equation analogous to (47) is obtained for the prelimit model. To 
this end, let 

(61) M n := e ■ Y n A e • Z n > 0. 

Denote also M n = n _1 / 2 AP. By (27) and (28) we can write e • X n = e ■ Y n - 
e ■ Z n and therefore by (61), 

(62) e-Y n = M n + {e-X n ) + , e ■ Z n = M n + (e • X n )~ . 

Let u n = Y n I (e • Y n ) when e • Y n > 0, and set u n = e\ otherwise. Similarly, 
v n = %nj( e . z n ) if e • Z n > and otherwise set v n = e I+1 . Letting U n = 
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(u n ,v n ), noting that U n takes values in U, and using Lemma 1, (27), (28), 
(45) and linearity of G on Dq, we have 

Y n = (M n + (e • X n ) + )u n , 

(63) 

Z n = (M n + (e ■ X n )-)v n 

and 

qr n = G(X n -Y n ,-Z n ) 
(64) = G(X n - (e • X n ) + u n , -(e ■ X n )~v n ) - M n G(u n , v n ) 

= G(X n ,U n ) - M n G(u n ,v n ). 

Defining 

bf(x, it) = - E rtAiix* u ) - e i( e • x ) +u * + 



Gy{u,v)=Y J tijG ij (u,v)-e? 



Uj 



we obtain from (30) 

x?(t)=x?> n + ri wr(t) 



+ 



o 



{V--J2^ij(Gij(X n (t), U n (t)) -M"(t)G i3 -K(t),« n (*))) 
-C(M"(tK(t) + (e-X"(t)) + nr(0) 



(is, 



hence 



X"(t)=l°'" + rr(i)+ / 6 n (X n (s),[/ n (s))ds 
(65) t 7 ° 

+ / M n (s)G n (n n (s),?; n (s))ds. 



o 

It is useful to note that 

\\b n (x,U) - b n i yi U)\\< Cl \\x -y\l 

(66) 

||6 n (x,[/)|| <ci(l+ ||x||), ra G N, [/ G U, X, y G M , 

where ci does not depend on n,U,x and y. 

Part (i) of the following result proves (37). Part (ii) regards the proposed 
P-SCP. 
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Lemma 3. Fix t. Assume that X n (t) G satisfies the inequality \\X n (t) — 
nx*\\ < olqu, where ao> is the constant from (36). Then the following con- 
clusions hold. 

(i) X n (t) G A" 7 . Namely, there exist quantities Y n (t), Z n (t), # n (i) sat- 
isfying (2)-(5) as well as (35). 

(ii) Let Y n (t) and Z n (t) be given, define 

(67) f n (t) = G{X n {t) - Y n (t),N n - Z n {t)) 

and assume that (4) and (35) are satisfied. Then (2), (3) and (5) also 
hold. 

Remark. The lemma has two implications. 

(a) Consider a jointly work-conserving P-SCP. If the inequality ||X n (i) — 
nx*\\ < a^n holds, then, by Lemma 3(i), X n (t) E X n . By definition of joint 
work conservation, a rearrangement of customers will be chosen so as to 
meet (35), and it follows that M n (t) = 0. Now, let r n = inf{s : M n (s) > 0}. 
The above discussion shows that if r n < t, then \\X n — nx*\\^ > a$n. In 
particular, \\X n — nx*\\*. n > oqu on {r n < oo}. This observation will be used 
in the next subsection. 

(b) p* is a legitimate sequence of jointly work-conserving P-SCPs. As 
argued in Section 2, one only has to show that when ||X n (t) — nx*\\ < aon, 

> holds. This is shown in Lemma 3(ii). 

PROOF of Lemma 3. Consider part (ii) first. By the definition of G 
(cf. Lemma 1), (2) and (3) hold. It remains to show that V?£-(i) > for all 
E £. By Lemma 1 and (17), 

(68) ip* = G(x*,u). 

Since it is assumed that M n (t) =0 [cf. (61)], (62) implies that ||F n (t)|| V 
||^ n (*)|| — ll^ n (^) ~~ nx * II • Hence by linearity of the map G on the domain 
D G and by (20), (44) and (36), 

(t) = Gij{nx\nv) + G i:j (X n (t) - nx* - Y n (t),N n -nv- Z n {t)) 

> ntfj - C G (\\X n (t) - nx* - Y n (t)\\ y \\N n - nv - Z n (t)\\) 

> niptj - 2C G \\X n (t) - nx*\\ - C G \\N n - nu\\ 

> nip*j — 2C G ctQn — cn 1 ^ 2 
>0, 

where the last inequality holds for all n large enough. 

To prove part (i), simply set Y n (t) = (e- X n (t))+e 1 , Z n (t) = (e- X n {t))~ e J+1 
and define ^ n (t) via (67). The result now follows from part (ii). □ 
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The following lemma refers to instantaneous routing through nonblocked 
activities in the construction of the N-SCPs p' of Section 2, by showing that 
one can find \I/ n meeting (58). 

Lemma 4. Let (ifr , X ,Y , Z) satisfy 
j 

(69) = 

i 

*y = 0, i ^ ii 

(70) ?i>0, 2j>0, i€l,j€j. 

Assume that all components of X, Y, Z and \P are integers. Let a subset 
8±c£ ("nonblocked" activities) be given. Then one can find (*&,X, Y, Z) 
satisfying relations analogous to (69), (70), and 

(71) X = X, Y <Y, Z<Z, 

(72) (i,j)e£i implies Yi A Zj = 0. 

Proof. Define inductively a sequence (x( fe ) , y ( fc ) , z( fc ) , ) , k = o, . . . , h , 
as follows. Let 

(X^,Y(°\zi°\^) = ^f,X,Y,Z). 

Let k > be given, for which (Jf( fe ), y( fc ), Z^ k \ is defined. If Y( fc ), 
satisfy (72), then terminate, declaring k\ = k. Otherwise, define (X^ k+1 \ 
y(k+i) ^ z (k+i) ^(fc+i)) as f n ows . Let i be the smallest iEl such that 

(k) (k) 

there is j with (z, j) G £\ and ly A > 0. Let jo be the smallest such j. 
For z G X, j G i7 define 

v-(M-l) _ yW _ y( fc ) _ ti .. . . 

7 (k+l) _ 7 (k) ^(k+l) _ ^(fe) 

Since by construction < e • Y( fc ) = e • Y — k, the procedure must terminate. 
Defining (X,Y,Z,W) = (X^, Y^\ Z^\ completes the proof. □ 

The following lemma will be useful in analyzing the N-SCPs p' . In order to 
show that ty n tracks closely estimates are required on ^"(i) — 1 f n ( 
The lemma shows that it suffices to estimate only (*$> n (t) — *f> n (t)y. 
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Lemma 5. Let (tp,x,y,z) satisfy 

3 

i 

tpij = 0, i ^ j, 

and let (ip,x,y,z) satisfy analogous relations. In addition, assume 

(73) if ' i ~ j and tpij < tpij then yi A Zj = 0, 
and yi>0, iEl, Zj > 0, j £ J . Then 

\ipij ~ 4>ij\ < c^2(ipij - tpij) + + c\\x - x\\, 

where c does not depend on ip,x,y, z,ip,x,y or z. 

Proof. Let e be an upper bound on tpij — tpij for all i, j, and on \xi — Xi\ 
for all i. Let jo be such that Zj = 0. Then 

2 = Yl fox + % > Yl > 

i i % 

and since ipij < tpij + e, tpij — ipij > —ce for every i ~ jo- Let iq be such 
that yi = 0. Then 

i j j 

Since V^oj < tfioj + e f° r every j, we have V'ioj ~~ ^ioj ^ — ce f° r every j ~ ?o- 
Thus we have shown that — V^jl < ce for every (z, j), i ~ j, with either 
yi = or zj = 0. In view of (73), we have shown that — < ce for 
every i ~ j, such that -0^- < tpij. On the other hand, if tpij > then 

simply \ipij — ipij\ = ipij — tpij < e by assumption. □ 

Denote 

(74) J? = \\Y t n -Y t n \\ + \\Z?-Z?\\, 

(75) g?(t)= f\ n (X^U^d S , Q n 2 {t)= f e^ s L(X2,U?)ds. 

Jo Jo 

Throughout, let p, p*, p' and ( be as in Theorem 2, and let / denote the 
unique Cp ol [C| under Assumption 3(iii)] solution to (49). Recall that J* f = 
f f(s)ds. 
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Proposition 1. Items (i)-(iii) below hold under p (in particular, under 
p* ) and under p' . 

(i) (X n ,Y n ,Z n ,V n )^(x*,0,0,ip*). 

(ii) (W n , JM n ,X n ,Q\,Q%) is tight. 

(iii) (W n ,^M n ) (W, 0), where W is a standard Brownian motion 
on M. 1 . Moreover, the following holds under p* and under p' : 

(iv) | J n \* s i — > and t — ► in distribution, for every < s < t < oo. 

For the proof see Section 3.2. 

Lemma 6. Under p and under p' one has the following. Denote by 
(X,Qi,Q2,W) a limit point of (X n ,Q 1 {,Q'!2,W n ) along a subsequence. Let (Ft) 
denote the filtration generated by (X,Qi,W). Then W is an (Ft) -standard 
Brownian motion, X , Q\ and Q2 have continuous sample paths, and Q\ 
has sample paths of bounded variation over finite time intervals. Moreover, 
J e~ ls Df(X™)-dQi(s) => / e~~ iS Df(X s ) -dQ\(s) along the subsequence, where 
f is the solution to ( 49 ). 

Proof. Based on Proposition 1, the proof of Lemma 6 is identical to 
that of Lemma 6 of [3] and is therefore omitted. □ 

Proposition 2. (i) Assume either case (i) or (ii) of Assumption 3 
holds. For either q=p, with fixed tuq S (m^,, 771^/2), or for q = p' , with fixed 
mo G (mL,mA(mA — 2)(5tua — 2) _1 ), one has 

(76) E«[(\\X n \\*) mo ]<C(l + t) mi 

where C, mi do not depend on n and t. In addition, the same conclusion 
holds in the case q=p'({h n }), for any sequence {h n }, and the constants C 
and m do not depend on the sequence {h n }. 
(ii) Let case (iii) of Assumption 3 hold. Then 

(77) E q ( [(\\X n \\*) mo ]<Ce Gt , 

for q =p'({h n }), mo G ((Xm^m^ — 2)(5m J 4 — 2) _1 ) ; and C not depending 
on n, t and the sequence {h n }. 

In part (ii) of the proposition, the significance is not in the precise form 
of the upper bound but in the uniformity with respect to {h n }. 
For the proof of Proposition 2 see Section 3.3. 

The method of [3], that we adopt here, is based on estimating the process 

(78) K t n = b(Xl\ Un ■ Df(Xn + L(A7, [?) - H(X?, Df(X?)) > 0, 
where the inequality above follows from (50). 
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Lemma 7. Let the assumptions of Theorem 2 hold. For every sequence 
p of admissible jointly work-conserving P-SCPs 

POO 

limmf / e^L^", U?) dt > V{x). 

J 

Moreover, if for some 5 > 0, T > and k G (0,oo], q is an admissible SCP 
under which 

(79) ^o P C {^n,k,T n e-^K?dt > 5 = 0, 



^n,fcT &em<? any event on which \\X n \\j, < fe, i/ien 

rT 



lim sup 



n— »oo 



l*w/ e-^L(X^,Undt 
Jo 



<V(x) + 5. 



Proof. Equipped with Proposition 1, Lemma 6 and Proposition 2 un- 
der Assumption 3(i) and (ii), and boundedness of V in case of Assump- 
tion 3 (hi), the proof is very similar to that of Theorem 4 of [3], and is 
therefore omitted. □ 

Proof of Theorem 2. Part (i): Established in Lemma 7. 

Parts (ii) and (hi): We use Lemma 7 with k = oo and £l n> k,T = fi. In view 
of this lemma it suffices to show that, for every T > 0, under both p* and 
p', Jq ' e~ ls 'Kg ds — > in probability. The function h* satisfies the following 
(see the proof of Theorem 1 of [2]): 

(80) H(x,Df(x)) = b(x,h*{x))-Df(x) + L(x,h*(x)), x€R T . 
Combining (78) and (80), 

K? = (b(X t n , Un - &(*?, h*{Xt))) ■ Df(X?) 

(81) 

+ L(X?,Un-L(X?,h*(X?)). 
By dehnition of b (46), and by (74), 

(82) \\b(X?, U^) - b(X?, h*(X?))\\ < cJ?. 
By (48) and (53), 

(83) L(A7, Un - L(X?, h*(Xn) = L(X?, ¥?(<)) - L(A?, ¥»(t)) 
where, using (63), 

:= G(*? - (e • X?)+v», -(e • X?)~v?) 

(84) = G(X? - Y t n + M?v$, -Z? + M?v?) 



30 



R. ATAR 



Note that 

(85) ||$? - *»|| = ||G(Xf - Y t \ -Z?) - G(X? - Y t \ -Z?)\\ < cJ?. 

By Assumption 2, L is uniformly continuous on compacts, hence there are 
functions a k (5) with lim^o ot k {5) = such that 

\L(x,ip) - L(x,ip')\ < a k (5), 

(86) 

||x|| V II^H V \\i/>'\\ < k, - tp '|| < 6. 

Combining (81)-(86), \\X?\\ V \\i>?\\ < k implies 

(87) A? < cJ?\\Df(X?)\\ + a ck (J? + M") < cJ?f3 k + a ck (J? + M?), 
where j3 k depends only on k. Since by (52), e • Y n A e • Z n = 0, we have 

(88) M n = e • Y n A e • Z n < \e ■ Y n - e ■ Y n \ + \e ■ Z n - e ■ Z n \ < J n . 
Moreover, by (27), (28) and (52), 

(89) J n < Cl (\\X n \\ + \\* n \\), 

for ci not depending on n. Fix T and let Q n > k > e ^ denote the event that 
\\X n \\* T V ||* n ||^ < k and \ J n + M n \l T < 5. By Proposition 1, 

(90) limliminf liminf liminf pi(n n ' k ' £ ' S ) = 1, 

k 5^0+ e^0+ n 
for q=p* and for q = p' . Combining (87)-(89), on £} n ' k ' e ' S we have 

0< / e^Kfdb 



(91) < c 2 e{kfi k + a C2k (c 2 k)) + c 2 T{(i k 5 + a C2k (6)) 
=: a(k, e, 5), 

where c 2 does not depend on n, e, 5, k. Taking^ ra — > oo, then e — > 0, 6 — > and 
finally k — > oo, using (90), it follows that / J e'^K^dt in probability. 
Since T is arbitrary, the result follows. 

Part (iv): Recall that the sequence p" = p'{{h n }) is defined like the se- 
quence p' , except that for each n 1 h is replaced by h n . It suffices to show 
that for each 5 > one can find h s such that 

roo _ 

(92) lim sup Ei / e^L^X?, U t n ) dt < V{x) + 45, 

n— >oo Jo 

where under q, h n are all equal to /i' 5 ^ because one can then find an appro- 
priate sequence 5 n — > and set /i n = /i" 5 ™ so as to obtain V((,p") < V(x). We 
will prove (92) by finding, for each 5 > 0, /i 5 and T such that 

(93) lim sup A? C V 7 *A(X t n , U?) dt < V(x) + 35 

n JO 
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and 

POO 

(94) sup.E5 / e-^LiX?, U?) dt < 5. 

n ^ JT 

Let 5 > be given. If L is bounded, (94) holds for T large. Otherwise, 
Assumption 3(i) or (ii) is in force, and (94) follows from Proposition 2(i) and 
Assumption 2(iv), for T large enough. In particular, by the last statement of 
Proposition 2(i), such T can be chosen independently of the sequence {h n }. 
Let such T be fixed. Using the two parts of Proposition 2 (ii) for the different 
cases of Assumption 3, one can find k £ (0, oo), not depending on {h n }, so 
large that 

(95) P^(\\X n \\* T >k)<5. 
In addition, in view of (90), for k large enough, 



l (nn , k , s ,s r j\-^L(X?,Undt 



<6. 



(96) lim sup lim inf lim inf E q r 

Let such k be fixed. 

We argue along the lines of the proof of Theorem 2 of [3] . From (80) , the 
function h* satisfies ip(x, h*{x)) = inf(/ e u<^(x, U) =: (f*(x), where 

tp(x, U) = b(x, U) ■ Df(x) + L(x, U). 

For each e > let d(x,y,e) denote the Euclidean distance from y to the 
boundary dB{x,eI 1 / 2 ). Let B Xji = eL 1 n B(x,el 1 / 2 ) and set 

d(x,y,e) 

y e B Xyg , 



d(x,y,e) = ^ Ey'eB x , e d ( x ^V^ £ Y 

0, otherwise. 



Let 



h s( x ) = d{x,y,e)h*(y). 

yeeZ 1 

It is elementary to check that x ^ hg{x) is Lipschitz. Since U *—> L(x,U) 
is convex by assumption, and U \— > b(x,U) affine, the map U i— ► <p(x,U) is 
convex. By Jensen's inequality and uniform continuity of (x, U) — ► f(x, U) 
and x i— > ip*(x) on B(0, k), for sufficiently small e > 0, 

b(x, he(x)) ■ Df(x) + L(x, he(x)) = ip(x, h g (x)) 

= V\^^d(x,y,e)h*(y)j 

<^2d(x,y,e)ip(x,h*(y)) 
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<Y,d(x,y,eMy,h*(y)) + 5/2 
y 

y 

< tp*(x) + 8 = H(x,Df(x)) + 8, 

x G B(0, k). 

Letting h s = tig we have [in place of (80)] 

(97) H(x,Df(x)) + 5>b(x,h S {x))-Df(x) + L{x,h\x)), xeB(0,k). 

Hence, arguing as in part (iii) above, recalling that h s is Lipschitz (a condi- 
tion used in the proof of Proposition 1) one obtains in place of (91), 

0< [ T e-~ /t K?dt<a(k,e,5) + 5, 
Jo 

holding on f2 n,fc ' e '' 5 . Taking e and 5 small enough so that a(k,e, 8) < 8, 
J^e-^K^dt < 25 on n n > k ' £ < s . Thus P(n n ' k ' £ ' 5 n {/ Q T e'^KJ 1 dt > 25}) con- 
verges to zero as n — > oo. By Lemma 7, 

lim sup E\ [ T e" 7i L(X i n , U?) dt 
n— >oo JO 



< V(x) + 25 + lim sup 

n— >oo 



l [nMr j o e-*L(X?,U?)dt 



Using (96), we obtain (93). □ 



Lemma 8. Let Assumption 4 hold. Then E(\\A n \\* t ) mA < c(l + t) mA l 2 , 
where c does not depend on n or t. 

For the proof see Lemma 2 of [3] . 

3.2. Tightness estimates. We prove Proposition 1. Most involved is the 
treatment of the nonpreemptive case. The main idea is a "bootstrap" ar- 
gument (a variation of which is also used in the next subsection), where 
one first establishes tightness of the processes up to a certain stopping time, 
and then uses this to show that the probability that the stopping time is 
incurred in an arbitrary fixed time approaches zero. The proof is established 
in a number of steps. 

Step 1. n~ l l 2 W n =► 0. 

Step 2. Under p, M n => and X n x* . Under p' , X n (- A a n ) x* , where 
a n = inf{s > : JM n {s) > 1} A 8 n . 
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Step 3. Under p, (X n , Y n , Z n , (x*, 0, 0, iff*), and (X n , W n , QJ, Qg) is 

tight. 

Step 4. Under p*, J n ^0. 

Step 5. Under p', conclusions of step 3 hold, upon stopping all processes 
involved at a n . As a result, J? M n =>• 0, and conclusions analogous 
to those of step 3 hold under p' . 

Sfep 1. Let Ai, iel, 5^, i ~ j, f?j, iel, be independent Brownian 
motions with zero mean and variance given by £Mf(l) = ^i^uii ESh(l) = 
fHj, ERf(l) = 6i. Set = for i ^ j, A = (Ai), S = (,%) and R = (Ri). For 
the fact (A n ,S n ,R n ) => (A,S,R) see [3], Lemma 4(i). Note that by (3) and 
(4), < iV} for every i,j and i. Moreover, Y?(t) < e • Y^(t) < n~ x [e ■ 

X n (0) + Af(t)] < c + Yh Af(t) =: C n (t), where c does not depend on 
n and t. Hence by (31), 

(98) ||^||**<ci||i n ||J+c 1 ||5 n ||^ + c 1 ||fl n ||Jn (t) , 
for a constant c\. From this it is elementary to show that 

(99) n'^W 11 ^ in distribution, as n^oo,t>0. 

Step 2. We show first that under p, M n =>- 0. Let r n = inf{s : M™ > 0}. 
We shall show that, for every T, P(r n < T) — > as n — > oo; this implies that, 
for every T, ||M n ||^ — > in distribution, and as a result M n 0. Indeed, 
by (65), 

/■tAr" 

X n {t A r n ) = l°' n + / & n (A n (s), U n {s)) ds + rr(f A T n ). 
Jo 

By (66), 

fiAr" 

\\X n (tAT n )\\<c 2 +C2 / (l + ||X n ( S )||)ds + C2 ||W"(tAr n )||, 

JO 

and it follows from Gronwall's inequality that 

(100) \\X n -x*\\* tATn =n~ 1 / 2 ||X n ||* Ar „ <c 3 n-^ 2 e C3t \\W n \\* t - 
Hence by Remark (b) following Lemma 3 and (99), 

(101) P(r n <T)< P(\\X n - x* \\* TAT n > oo) - 0. 

As a result, M n 0. By (99), (100) and (101) it follows that for every t, 
\\X n — x*\\* — > in distribution as n — > oo. As a result X n =>■ x* . 
Next, under p' , recall that n is defined in (59) and let 

(102) cJ n = inf{t>0:^Ar(t)>l}Ae n . 
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By (65) and (66), 

\\X n (t A cr n )|| <c 4 + c 4 / (l + \\X n (s)\\)ds + c 4 \\W n (tAa n )\\. 

J 

Using again Gronwall's lemma and (99), we have 

(103) \\X n - nx* \\t Ar7 n in distribution, t > 0. 

Step 3. This step refers to p only. By (62), 

e ■ Y n = n- l ' 2 {M n + (e • X n ) + ) < n - x l 2 M n + \\X n - x% 

Since Yi > 0, i £2, it follows that Y n =>• 0. By a similar argument, Z™ => 0. 
By (67), (68) and linearity of the map G on Dq, 

V n = n - l G{X n - Y n , N n - Z n ) 

= G{X n -Y n ,n^N n -Z n ) 

= G(X n -x* - Y n , n^N n -u-Z n ) + G(x*,u) 

= G(X n -x* - Y n , n~ l N n -v-Z n ) + iP*. 

Since Y n ,Z n ,(X n — x*) =>• and by (20) and continuity of G, we obtain 

We have now shown that \\X n - x*\\* + \\Y n \\* + \\Z™\\* t + ||K n - ip*\\* t 
converges to zero in distribution, for every t. Hence (X n ,Y n ,Z n ,^ n ) =>■ 

Next we show that the sequence (X n , W n , Q1, Q%) is tight in (D(R fc )) 3 x 
D(R). We have shown already that S n =>S, R n ^ R, ^ n =► V* and Y n => 0. 
An application of the time change lemma [5] shows that S^(Jq ^fj(s) ds) => 

Sij(ijj*-) and Rf (J Q Yf(s) ds) 0. By (31) and (33) it follows that W n => W, 
a standard Brownian motion in M. 1 . 
By (65), (66), 

\\X n (t)\\ < \\X°^+c 5 \\W n (t)\\+c 5 J?M n (t) + c 5 [ (l + \\X n (s)\\)ds 

Jo 

and therefore by Gronwall's inequality, 

(104) HXl* < c 6 e C6 *(l + + \\W n \\). 
Using tightness of W n and M n , it follows that for every t, 

(105) lim limsupP(||i" n ||j? > m) = 0. 

m— >oo »oo 

Fix T. By (65) and (66), for s,t € [0,T], s < t, 

\\X n (t) - X n (s)\\ < c 7 \\W n (t) - W n (s)\\ 

(!06) 

+ c 7 (t-s)(l + \M n \* T + \\X n \\* T ). 
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Let w(x,S) = sup s teS \\x(s) — x(t)\\ where S C [0,T), and let 
w' T (x,5) = inf maxw(x, [ti-i,U)), 

l<i<v 

where the infimum is over all decompositions 1 < i < v, of [0,T) 

such that U — ti-i > 5, l<i<v (cf. [5], page 171). The notation 

(107) wt(x,5)= sup w(x, [s,t]) 

0<s<t<(s+8)AT 

will also be useful. It follows from Theorem 16.8 of [5] and tightness of W n 
that for each t and e, lim^ nmsu Pn-»oo P(w' t (W n , 5) > e) = 0. Hence by 
tightness of M n and using (106), for each t <T and e, 

(108) lhnlimsupP(^(i" n ,<5) >e) = 0. 

<5— >0 n— >oo 

Since T is arbitrary, (105) and (108) imply tightness of X n , using Theo- 
rem 16.8 of [5]. By (66), (75) and Assumption 2, there is a constant eg such 
that 

\\QW t vm\*t<c8t(i+\\x n \\* t r\ 

and for s,t £ [0, T], s < t, 

WQUt) - QUs)\\ V \Q n 2 {t) - Q n 2 (s)\ < c 8 (t - s)(l + ||X n ||^. 

Hence, using Theorem 16.8 of [5], tightness of Q\ and Q 2 follows from (105). 
We have shown that X n , W n , Q™ and Q 2 are tight, and using Proposi- 
tion 3.2.4 of [6], it follows that (X n , W n , Q?, Q%) is tight. 

Step 4. Fix T. By (1) and (55), under p*_ one has | J n \* T < 2(1 + J)n~ 1 / 2 
on the event {\\X n — x*\\^ < ao}. Since \\X n — x*\\^ converges to zero in 
distribution, so does | J n |^, and since T is arbitrary, J n 0. 

Step 5. This step refers to p' . Let a n be as in (102) and recall (103). 
Reviewing step 3 shows that all its conclusions still hold under p' in place 
of p, upon replacing X n by X n (- A a n ), Y n by Y n (- A a n ), and similar sub- 
stitutions for the processes W n , X n , Qi and Q 2 . As a result, 

(109) {X\ an , Y^n , Z? Aan , *? A(T „ ) (x*, 0, 0, r ), 

(110) (X^^W^^QU- A<7 B ),Q5(- Aa n )) is tight. 
Let T be fixed and denote T n = T Aa n . For i ~ j let 

(111) A£ = *£-% 
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Let fi n,fc denote the event 

{\\X n \\* T n V \\Y n \\* T n V \\Z n \\* Tn V \\^ n \\* T n < k}. 
By tightness of X n (- A a n ) and by (62), (64), 

(112) lim liminf P(0 n ' fc ) = 1. 

We will show that 

(113) lim lim sup P(cr n < e ) = 0, 

£0^0+ n 

and that for every i ~ j and small enough eq > 0, 



(114) lim sup P a n > e , sup A™- > ce = 0. 

n V [e ,T»] / 

To this end let £o £ (0, T). Fix z ~ j. If A™- > on [s,r), no customers 
are routed on activity on this time interval and by (57) and (9), 



= *£( S )-A£( S ,t), te[s,r) 



where 



(115) A%(s,t) = 5™(^.(t)) - S^^(s)). 
Therefore 

(116) ^(t) = ^(s)-n~y^ j (s,t). 
Now, 

(117) p(a n >e , sup A™ (i) > 3e) < P((tt n ' fc ) c ) + P(ft"' fe ) + P(^' fc ), 
where 

n n,k = Qn ,k n > £q | 

n(30<s<r<r n :A",(s)<e, inf A£(<) >e,A£(r) > 3el, 

I <G(s,r) ^ J 

f^' fc = Cr^n{a n >£o}n( mf A™- > el. 

Let k be fixed. By the Holder assumption on hi, hi away from e • X n = 0, 
there are positive constants c' k , (depending on k and e but not on n) 

such that the following holds on fi n ' fc : 

m(t) - = \G tJ (x n (t) - Y n (t), ~z n (t)) 

- Gt ,(X n (s)-Y n (s),-Z n (s))\ 
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< c(\\(X n (t) - Y n (t)) - (X n (s) - Y n (s))\\ + \\Z n (t) - Z n (s)\\) 

(H8) . , 

< c' k (\\X n (t) - X n (s)\\ + \\X n (t) - X n (s)\n 

e e 

+ ^{Wm^f*} + 4 1 {|e-Xf|<e/8} 
<C k \\X n {t)-X n (s)\r + £ -. 

By (23), for < s < t < r < T n as above, 

n-^A^t) = - S^ljis)) 

(119) +n 1/2 ^- A^W-V* )^ 

J s 

+ n 1 / 2 ^ - fOjWfi -s) + n^fH^t - s). 

Since < N™, it follows that are a H bounded uniformly by a constant. 

Since the jumps of the process are all of size n -1 / 2 , a use of (12.9) of [5] 

shows that \S%(J?mt)) - S%{JP%{s))\ < 2w' C9Tn (SJ l J ,c 9 (t - s)) + n' 1 / 2 , 

where eg > is a constant. Since \\^ n — ^*||^n < kn" 1 / 2 on Q n ' k , using (19), 
and assuming n is large enough we therefore have 

(120) n-^Af^t) > -2w' CgTn (S^,c 9 (t - a)) - rT 1 / 2 + c w n l ' 2 (t - s), 
where cio = fJ.ijip^j/2. Hence on noting that A^-(s) < e, 

A£(i) = A^-(s) + (*£(t) - - (¥g.(t) - ( s )) 

< -n- l l 2 A^(s,t) + c k \\X n {t) - X n {s)\r + 2e 

< 2w' C9Tn (S? j ,c 9 (t - a)) - c 10 n l l 2 {t -s) + c k \\X n (t) - X n (s)r* + 2e. 
Since on Q?' fe , A" (r) >3e, 

£ <2< Tn (S5,c 9 (r- S )) 

(121) - Qon 1 / 2 ^ - s ) + c fc ||X» - X n (s)f>\ 

p{^ k )<p{vq M ) + p(yii M ), 

where 

fi3 >fc = {30<s<r <T n :r- s<an- 1/2 , (121) holds}, 
n£' fe = {3 0<s<r<T n :r-s> an~ 1/2 , (121) holds}. 

On Og'^, 

e < 2u;' ^(S^cgan" 1 / 2 ) +c fc w^ l (l n ,an- 1 / 2 ) Pfc 
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and since (S n ,X n (- Aa n )) are tight, for every a, 

limP(ft"' fc )=0. 

On ' fc , 

cioa<2||^||: 9T „ + c fc (2||l"||^) Pfc 
and by tightness of (S n ,X n (- A a n )), 

lim limsupP(Q4' fc ) = 0. 

a >oo n — >00 

As a result we have, for every k, 

(122) lim P(0"' fc ) = 0. 

n— >oo 

Regarding f^'^ substituting s = and i G [0, £q] in (118)-(120), 



A£(t) < A$(0) + 2||Sg||* 9£0 - don 1 ^ + c fc ||X»(*) - + e/2. 

Since A^-(eo) > £, we have on U 1 ^' 

e/2 < A«.(0) + 2115™ ||* - don 1 / 2 ^ + c k \\X"(e ) - X ^ ■ 



By tightness of the random variables £ n := 2||S , |J-||^ geo + Cjt||X n (eo A a n ) — 

X°' n || Pfe , the fact that a n > Eq on O^' , and since A™(0) are bounded [as 
follows from (21), (22)], 

(123) limsupP(^2' fc ) < limsupP(£ n > ceon 1 / 2 ) = 0. 

n— >oo n— >oo 

Combining (117), (122), (123) and (112), we obtain (114). 

Now, on n n > k , M n (a n Ae )<e- Y n (a n A e ) < k. Hence by (102), 

P(* n < so) < P(J?M n (a n A e ) > 1) < P((n n ' k ) c ) + t {£ok > 1} . 

Therefore, for every k, 

lim sup lim sup P(a n < e ) < limsupP((ft n ' fc ) c ), 

£0^0+ n n 

and (113) follows using (112). 

Having established (113) and (114), we argue as follows. Lemma 5 and 
the property (58) of the policy p' imply that 

(124) ||A^)||< C11 £(A^))+ 
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By (53) and the uniqueness statement in Lemma 1, Xf — Y™ = Y^,j f° r 
every i. This and (27), along with an analogous argument for Z n , imply that 

(125) \\Y n - Y n \\ + \\Z n - Z n \\ < ci 2 \\A n \\. 
Combining (88), (124) and (125), on (l n ' k , for every t G (e ,T], 

(126) JM n (t) < c 13 ke + c 13 tJ2 sup(A^) + . 

i~j [eo,*] 

Hence, with bo as in (60), 

P(a n <T)< P({n n ' k ) c ) + P{a n < e„) 

+ P(a n >e ,^M n (T n )>l) 



(127) 



+ P[a n > e ,max sup (Af-) > b 

*~i [eo,T«] 



<P{{n n > k ) c ) + P(a n <e ) 



+ P[a n >e Ql c 13 TY, sup (A™-) + > 1 - c 13 ke ) 

\ i^j [e 0l T»] / 

+ P[ max sup (A™ ) + > 6 

V*~J [ £0 ,T»] 



Taking eo small enough and using (112), (113) and (114), we have that 

(128) limP(<7 n < T) = 0. 

Since T is arbitrary, we finally have from (109), (110) and (128) that 
{X n ,Y n ,Z n ,^ n )^(x*,0,0,ij*), (l n ,PF\Q?,Qf) is tight. 
Also, with (128), the relations (114) and (124) show that 

(129) limPf sup ||A n || >ce) =0. 

71 V[ £0 ,T] / 

In view of (74) and (125), we have from (129) that |«/ n |st converges to 
zero in distribution, for every < s < i < oo. As follows from (88), a simi- 
lar statement holds for |M n |* 4 . Moreover, using again (126), now equipped 
with (129), letting n — > oo, then eo — > + and finally k — ► oo, we obtain that 
j?M n (T) — > in distribution, and since T is arbitrary, J^M n =>• 0. This 
completes the proof of Proposition 1. 
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3.3. Large time estimates. In this section we prove Proposition 2. The 
following estimate from [2] is used in a crucial way. 

Proposition 3. Let Assumption 3(i) or (ii) hold. Let (27)-(S0) hold. 
Then 

(130) \\X n (t)\\ <C(l + t) m (\\x\\ + \\W n \\* t +\M n \* t ), 

for C and m depending only on the model parameters, and in particular, not 
depending on x, W n or on the SCP. 

Proof. We use the following result from [2]: If 
Xi = Wi -^fiijJfipij - Qi^yi, 

(131) 

ip = G(x — y, — z), e ■ y A e • z = 0, 

then 

(132) \\x\\ <c{l+t) m \\w\\* t , 

where the constants c, m do not depend on tp,x,y,z: Under Assumption 3(i), 
(132) follows from the proof of Corollary 1 of [2]. Under Assumption 3 (ii) , 

(132) follows from Theorem 3 and Lemma 3 of [2]. 
Let 

(133) W?(t) =W n {t) + r~ l f ' M n {s)G n {u n {s),v n {s))ds. 

Jo 

We have from (65) X t n = X°> n + J* b n (A* , ds + rW?(t) . Lemma 2 implies 
that there are Y£ and Z r { such that (38)-(41) hold. Hence X n ,Y[\Z'{, 
WI, Wi satisfy relations analogous to (131). As a result, a relation as in (132) 
holds, and using (133) we obtain (130). □ 

Remark. The only two places in this paper where Assumption 3(i), (ii) 
is used are in Theorem 1 and in obtaining (132) from (131). In fact, also the 
proof of Theorem 1 (carried out in [2]) uses this assumption only in order 
to establish (132), and if the implication "(131) implies (132)" holds true in 
greater generality, then so do Theorems 1 and 2. 

Proof of Proposition 2. Most of the proof deals with part (i). Let 
Assumption 3(i) or (ii) hold. First consider policies of the form p. By (98), 

\\w n \\* t < cl (\\A n \\* t + \\s n r clt +\\R n \\* CHt) ). 

By Assumption 4, applying Lemma 8 shows that, for an appropriate con- 
stant C2, £ , (||^4 n ||J t ) mA < 02(1 + t) C2 , and a similar estimate holds for S n . 
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Conditioning first on Q n and using the independence of A n and R n , a simi- 
lar estimate follows for the last term above. As a result, 

(134) E(\\W n \\* t ) mA <c 2 (l+t) c \ 

As in the proof of Proposition 1, let r n = inf{i :M t n > 0}. Under {r n > t}, 
\M n \t = and therefore by Proposition 3, 

(135) \\X?\\* <C(l + t) m (\\x\\ + \\W n \\;), {r n >t}. 
On {r n < t}, by (3), (5) and (20), 

\M n \* t = n- l / 2 \M n \* t < n~ l ' 2 \e ■ Z n \*t < n~ 1/2 e • N n < C377 1 / 2 , 
and by Proposition 3, 

(136) ||X"||* <C{l + t) m (\\x\\ + WWm + n 1 ' 2 ), {r n <t}. 

Combining (134), (135) and Lemma 3 [in particular, Remark (a) that fol- 
lows] , 

P{r n <t)<P{\\X n \\l hTn >a ny 2 ) 

(137) <cn- mA / 2 E(\\X n \\* tATn ) mA 

<c A n- mA l 2 {l+tf\ 

Therefore by (136), (137) and the Holder inequality, with q~ l + q~ x = 1 and 

qm = m A , 

E(\\x^;r° <E[(\\x^;r°t {T n >ty ] 

+ {E(\\X n \\* t ) mA ) 1/q {P{T n < t)) 1/9 " 

(138) 

< c 5 (l + t) C5 + c 5 n^ lmA n~ {2 ^ mA (l + t) C4 ^ 
<c 6 (l + t) C6 , 

where the inequality q~ l — q~ l < 0, used on the last line above, follows from 

777-0 < 777yl/2. 

Next consider the policy p'. Let b\ := (2&o) V 13, where bo is the constant 
from (60). Let 



t?„ = inf <j t > : max A"- > bi }. 



By (88), (124) and (125), 



M n < J n < c 7 \\A n \\ < c 8 max(A™,; 



Letting T n = T A 7? n , it follows that 

\M n \*T<csh. 
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Hence by Proposition 3, we have 

\\X n \\* Tn <C(l + T n ) m (\\x\\ + \\W n \\* Tn +c 8 b 1 ) 

(139) 

< C9 (i + ry m (i + ||^||^), 

where eg depends on x but not on T or n. We establish below the estimate 

(140) P($ n <T)< con 1 / 4 -" 1 - 4 / 8 ^ + T) m , 

where cq and m are constants that do not depend on T or n. Repeating 
the argument of (138), with i9 n in place of r n , and (139) [resp., (140)] in 
place of (135) [resp., (137)], shows that E(\\X n \\^) m ° < c w (l + T) Cl °, with 
qmo = rriA, provided that ttiq < mA(m,A — 2)(5m j 4 — 2) . 

In what follows we prove (140). The argument is similar to that used to 
prove tightness in Section 3.2, but since the estimates must be uniform in 
time, a more careful analysis is required. By (62) and (139), 

\\X n \\*T n , \\Y n \\T n , U^llT n> ll*lr„ < cn(l +T) m (l + \\W n \\* T ) =: £(n,T). 
Clearly 

P{$n < T) <J2 P ( SU P Kj(t)>h). 

iZj Vi < T J 

Let i ~ j be fixed. Recall that bi/2 > b . As a result of (56)-(60), 
p(supA£(i)>&i) 

(141) < P(3 8 , r £ [0 n , T n ],s < r : A% (s) < h/2, 

/§(t)>0,te[8,r],A? j (r)>b 1 ). 

Note that b\ was chosen so that 1 + max^j sup n A^-(0) < &i/2. Note also 
that with A™ as in (115), (119) still holds. Arguing as in (118), using the 
fact that for s > n , h(t, ■) = h°, a constant function, with an appropriate 
constant c\2 one has 

(142) \%(t) - %(s)\ < c l2 i{n,T)\\X n {t) - X n {s)\\ + 1. 

Also, ^ n - i>* = n~ l /H n and therefore \\^ n - V*|It„ < n _1 / 2 C(n,T). Thus 
for s,t as in (141), recalling the notation (107) for wt(x,S), using (119), 

n-^^is^y -2w Cl3Tn (S^,c 13 (t- s)) 

(143) 

- c 13 £(n,T)(t - s) + cn l l 2 {t - s), 
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with c = mm > 0. Combining (116), (111), (142), (143) and specializ- 
ing to t = r, 

<2w Cl3Tn (S? j ,c 13 (t-s)) 

+ c 13 £(n, T)(t - s) - cn l l 2 (t - s) 
+ c 12 i(n,T)\\X n (t)-X n {s)\\. 



(144) 



Hence 



where 



p(0n<T)<p(n?) + p(n2), 

fi? = {3 < s <t <T n :t - s < ra~ 1/4 , (144) holds}, 
OJ? = {30< s<t <T„:t-s>ra~ 1/4 , (144) holds}. 

On o™, 

6i/2 < 2w CiaTn (% rT 1 / 4 ) + Cl3 ^(n, T)n -1 / 4 + c 12 £(n, T) ||l t " - X?\\ , 
where t — s < ra" 1 / 4 . By Lemma 9 below, recalling that b\ > 12, 

By (65), and the bound b x on M n , we have \X n (t) - X n (s)\\ < c£(n,T)(t - 
s) + c||M^ n (t) — W n (s) || +bi(t — s). Arguing again by Lemma 9, choosing @q 
large enough, 

P(ci 2 £(n, T) max{ \\X? - X s n || : < t - s < n~ 1/4 , t<T n }> &i/6) 

Also, by (98) and Lemma 8, 

P{i{n,T)rT x ' 4 > 6i/6) < P(||^1t > cn 1/4 ) < ci 6 (l + r) m ^ 2 rr mA / 4 . 
Hence 

(145) P(ny) < CisT^yi 1 / 4 "" 1 - 4 / 8 + c 16 (l + T) mA//2 n~' mA / 4 . 

On nj, 

^ 1/4 < H5?illc ls T n + ci3C(n,T)P„ + Cl2 £(n,T)||Xf - 

<n^ii; 3 T n +ci7(i+p) ci7 (i+iiwi^) 2 - 

Hence by (134), 

(146) P(n%) < ci 8 (l + T) Cl8 n-"^/ 8 . 
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Combining (145) and (146) we obtain (140). Finally, the last statement in 
the proposition follows since all estimates used in this proof do not depend 
on the function h n : we have used the fact that for t > n , h(t,-) =h°. 

Part (ii). Using (65), (66) and Gronwall's inequality [just like in the deriva- 
tion of (104)], one has for q = p'({h n }) and an appropriate constant C\ not 
depending on t, n and {h n } 

\\X n \\* t < C ie Cl \l + \\W n \\* t + |M n | t *). 

Replacing the estimate (130) with the above and reviewing the proof above 
of part (i), one recovers the estimate (77) as claimed. □ 

Lemma 9. Given (3q>0 there are constants c±,Pi, independent of n and 
T such that 

P(u; T (i",n- 1/4 ) > T~^) < ciT^n 1 / 4 "" 1 - 4 / 8 , n G N,T > 1. 
A similar estimate holds for Sfj in place of Af . 

Proof. Fix i and suppress it from the notation. By (23), 

A n (t) - A n {s) = n- 1 / 2 ^^) - A n {s)) - n-^A^t - s). 

Thus 

(147) PiwriA"',^ 1 ^) > T-P°) < P(A n (T) > 2\ n T) + P(Q^ + ) + P(Q,q_), 
where 

n™ + = {A n {T) < 2A n T, 3s,t£ [0, 2A n T] , < t - s < n" 1 / 4 , 

A n (t) - A n {s) > n^T" 130 + X n (t - a)}, 

n» _ = {A n {T) < 2A n T, 3 s, t € [0, 2A n T] , < t - s < n" 1 / 4 , 

A n {t) - A n (s) < -n^T-? + X n (t - s)}. 

Recall that EU{k) = 1. Letting U n (k) = U n (k) - (A™)" 1 , by (6) we have 
EU n (k) = 0. Let Mf = Y?k=i U n {k) and note that it is a martingale. For 
a real-valued function X on Z + let osc(X, i,j) =max{|X(A;) — X(l)\ :k,l£ 
[i + l,j]}. By (7), using A n < C2n, denoting p = C2n 3 / 4 , we have 

P(n% i+ )<P 3j<2A n T,3r<?i- 1 / 4 A n , ^ £/"(&)< (A n )~V 

V k=j+l ) 

<P 3j<2c 2 nT,3r<c 2 n 3 / 4 , £ < -(A n )~ V^r"* 

V fc=jr + l / 
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< P(3 j < 2c 2 nT,osc(M n ,j,j + 2c 2 n 3/4 ) > 1 n^ 1/2 T~ /3 °) 
<P{3je[0,2c 2 nT}D{0,p,2p,...}, 

o S c(M n ,j,j + P )> C2 1 n- 1/2 T-^/3) 

< 1 - (1 - P(\M n \* > c^n-WT-h /6)) 2nV4T '. 
Burkholder's inequality shows that 

E(|M n |p m ^ < c 3 E(p\U(l)\ 2 ) mA/2 < c 4 n 3m - 4 / 8 (A n )- m ^ < csn" 5 "^ 8 . 
Hence 

-P(^0,+) < 1 - (1 - c 6 n- mA/8 T mA/3o ) 2nl/4T < C7n l/4-m A /8 T l+m A /3 0; 

for an appropriate constant cj. By a similar argument one shows that a 
similar bound holds for P{Q^J). As follows from Lemma 8, 

P{A n {T) > 2\ n T) < cn- m ^ 2 {\ + T) _ "W 2 . 

Hence by (147), 

P{w T {A n ,n^) > 1) < Cin l/4-mA/8 r l+mAft 

for an appropriate constant c\ independent of n and T > 1. Since the Poisson 
processes S,™ are in particular renewal processes (with finite m^th moment 

for interarrival times), a similar result holds for Sfy □ 

APPENDIX 

Derivation of (30). By (24), (21), (23) 
X?{t) = n-y 2 {X?{t)-nxt) 

= X°' n +ii?(t)-E S 5(jf 



+ n 



-1/2 



i7»( s )d8 



I j Jo 

Substituting (31), (19), (14) and (26) in the above, we obtain 

xm = x°> n + n wm - 2 ^. *-.( S ) d S 



1/2 



Finally, according to (14), (17) and (19), the last term above can be written 
as — J2j p^ipijt. Equation (30) follows. □ 
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